1. Technical Field
The present invention is generally directed to an improved data processing system. More specifically, the present invention is directed to a system and method for maintaining checkpoints of keyed data structures using a sequential log.
2. Description of Related Art
Many application processes need a mechanism to store their local state in persistent storage to ensure that data is available across process crashes. If the states consist of a collection of key-value pairs, then a database system, such as DB2, Oracle, MySQL, etc., can be used to store the local state of the application processes. In such cases, the database system stores the local state of a process every time the state of the process changes. Thus, all intermittent values of key-value pairs associated with a process are stored in the database system.
However, in some cases, the local state may only need to be stored intermittently for performance reasons. In such cases, “soft checkpoints” may be generated in order to provide a point in time at which the local state of a process was known and events occurring after the “soft checkpoint” may be replayed in order to restore the current local state of the process.
An example of where a soft checkpoint is used is where there is already a persistent log from the beginning. By taking a soft checkpoint, the state can be reconstructed more quickly by restoring the checkpoint and replaying only the input events subsequent to the checkpoint. Therefore, the soft check point does not add to the resilience of the application, but allows event logs to be reclaimed and reduces the amount of work needed to restore the local state. With soft checkpoints, the data need not be saved persistently every time it changes and is rarely read since it is only needed to reduce the work needed to recover from a failure.
The usual database solutions for storing local states of processes are designed to optimize performance for high rates of read queries. Since soft checkpoints are only performed intermittently and are rarely read, known database solutions are not an optimum solution for storing local states of processes. Therefore, it would be beneficial to have an improved system and method for maintaining checkpoints of a keyed data structure, optimized for reducing the cost of updating the checkpoint during failure-free operation.